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jemalloc: You are probably already using it 

jemalloc is a userland memory allocator that is being increasingly adopted by software projects as a 
high performance heap manager. It is used in Mozilla Firefox for the Windows, Mac OS X and Linux 
platforms, and as the default system allocator on the FreeBSD and NetBSD operating systems. 
Facebook also uses jemalloc in various components to handle the load of its web services. However, 
despite such widespread use, there is no work on the exploitation of jemalloc. 

Our research addresses this. We begin by examining the architecture of the jemalloc heap manager 
and its internal concepts, while focusing on identifying possible attack vectors, jemalloc does not 
utilize concepts such as 'unlinking' or 'frontlinking' that have been used extensively in the past to 
undermine the security of other allocators. Therefore, we develop novel exploitation approaches and 
primitives that can be used to attack jemalloc heap corruption vulnerabilities. As a case study, we 
investigate Mozilla Firefox and demonstrate the impact of our developed exploitation primitives on the 
browser's heap. In order to aid the researchers willing to continue our work, we have developed a 
jemalloc debugging tool (named unmaskjemalloc) for GDB using its support for Python scripting. 

jemalloc Technical Overview 

jemalloc recognizes that minimal page utilization is no longer the most critical feature. Instead it 
focuses on enhanced performance in retrieving data from the RAM. Based on the principle of locality 
which states that items that are allocated together are also used together, jemalloc tries to situate 
allocations contiguously in memory. Another fundamental design choice of jemalloc is its support for 
SMP systems and multi-threaded applications by trying to avoid lock contention problems between 
many simultaneously running threads. This is achieved by using many 'arenas' and the first time a 
thread calls into the memory allocator (for example by calling malloc(3)) it is associated with a 
specific arena. The assignment of threads to arenas happens with three possible algorithms: 

1. with a simple hashing on the thread's ID if TLS is available 
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2. with a simple builtin linear congruential pseudo random number generator in case 
MALLOC_BALANCE is defined and TLS is not available 

3. or with the traditional round-robin algorithm. 

For the later two cases, the association between a thread and an arena doesn't stay the same for 
the whole life of the thread. 

Continuing our high-level overview of the main jemalloc structures, we have the concept of 'chunks', 
jemalloc divides memory into chunks, always of the same size, and uses these chunks to store all of 
its other data structures (and user-requested memory as well). Chunks are further divided into 'runs' 
that are responsible for requests/allocations up to certain sizes. A run keeps track of free and used 
'regions' of these sizes. Regions are the heap items returned on user allocations (e.g. malloc(3) 
calls). Finally, each run is associated with a 'bin'. Bins are responsible for storing structures (trees) of 
free regions. Figure 1 illustrates in an abstract manner the relationships between the basic building 
blocks of jemalloc. 
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Chunks 

In the context of jemalloc, chunks are big virtual memory areas that available memory is conceptually 
divided into. As we have mentioned, chunks are always of the same size. However, each different 
jemalloc version has a specific chunk size. For example, the jemalloc version used in Mozilla Firefox 
has a chunk size of 1 MB, while that used in the FreeBSD libc has a chunk size of 2 MB. Chunks are 
described by 'arena_chunk_t' structures, illustrated in Figure 2. 


/* Arena chunk header. */ 

typedef struct arena chunk s arena chunk t; 
struct arena_chunk_s { 

/* Arena that owns the chunk. */ 
arena t * arena ; 

/* Linkage for the arena’s chunks_dirty tree. */ 
rb_node (arena chunk t ) link_dirty; 

#ifdef MALLOC _ DOUBLE _ PURGE 

/* If we’re double-purging, we maintain a linked list of chunks which 

* have pages which have been madvise(MADV_FREE)’d but not explicitly 

* purged. 

* We’re currently lazy and don’t remove a chunk from this list when 

* all its madvised pages are recommitted. */ 

LinkedList chunks_madvised_elem; 

iendif 

/* Number of dirty pages. */ 
size_t ndirty; 

/* Map of pages within chunk that keeps track of free/large/small. */ 
arenachunkmapt map [1]; /* Dynamically sized. */ 

} ; 

Figure 2: Chunks [arena_chunk_t] 

Arenas 

An arena is a structure that manages the memory areas jemalloc divides into chunks and the 
underlying pages. Arenas can span more than one chunk, and depending on the size of the chunks, 
more than one page as well. As we have already mentioned, arenas are used to mitigate lock 
contention problems between threads. Therefore, allocations and deallocations from a thread always 
happen on the same arena. Theoretically, the number of arenas is in direct relation to the need for 
concurrency in memory allocation. In practice the number of arenas depends on the jemalloc variant 
we deal with. For example, in Firefox's jemalloc there is only one arena. In the case of single-CPU 
systems there is also only one arena. In SMP systems the number of arenas is equal to either two (in 
FreeBSD 8.2) or four (in the standalone variant) times the number of available CPU cores. Of course, 
there is always at least one arena. Arenas are described by the structure illustrated in Figure 3. 
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#ifdef MALLOC 


/* All operat. 

#ifdef MO 2_MEMORY 

malloc_spinlo' 


#ifdef MALLOC_ST AT S 

arena_st at s_t 

#endif 


lock:; 
lock: ; 


manages. */ 


bins [1]; / * Dynamically 

Figure 3: Arenas (arena_t) 


Runs 


Runs are further memory denominations of the memory divided by jemalloc into chunks. Runs exist 
only for small and large allocations (size classes are explained in the next paragraph], but not for 
huge allocations. In essence, a chunk is broken into several runs. Each run is actually a set of one or 
more contiguous pages (but a run cannot be smaller than one page). Therefore, they are aligned to 
multiples of the page size. The runs themselves may be non-contiguous but they are as close as 
possible due to the tree search heuristics implemented by jemalloc. 


The main responsibility of a run is to keep track of the state (i.e. free or used) of end user memory 
allocations, or regions as these are called in jemalloc terminology. Each run holds regions of a 
specific size (however within the small and large size classes as we have mentioned) and their state 
is tracked with a bitmask. This bitmask is part of a run's metadata; these metadata are portrayed in 
Figure 4. 


typedef struct arena _ run _ s are 
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Figure 4: Runs [arena_run_t] 


census 


Black Hat USA 2012 


4 







Exploiting the jemalloc Memory Allocator: Owning Firefox's Heap 


Regions 

In jemalloc the term 'regions' applies to the end user memory areas returned by malloc(3). As we 
have briefly mentioned earlier, regions are divided into three classes according to their size, namely: 

1. small/medium, 

2. large and 

3. huge. 

Huge regions are considered those that are bigger than the chunk size minus the size of some 
jemalloc headers. For example, in the case that the chunk size is 4 MB (4096 KB) then a huge 
region is an allocation greater than 4078 KB. Small/medium are the regions that are smaller than a 
page. Large are the regions that are smaller than the huge regions (chunk size minus some headers) 
and also larger than the small/medium regions (page size). 

Huge regions have their own metadata and are managed separately from small/medium and large 
regions. Specifically, they are managed by a global to the allocator red-black tree and they have their 
own dedicated and contiguous chunks. Large regions have their own runs, that is each large 
allocation has a dedicated run. Their metadata are situated on the corresponding arena chunk 
header. Small/medium regions are placed on different runs according to their specific size. As we 
have explained, each run has its own header in which there is a bitmask array specifying the free and 
the used regions in the run. 

Bins 

Bins are used by jemalloc to store free regions. Bins organize the free regions via runs and also keep 
metadata about their regions, like for example the size class, the total number of regions, etc. A 
specific bin may be associated with several runs, however a specific run can only be associated with 
a specific bin, i.e. there is an one-to-many correspondence between bins and runs. Bins have their 
associated runs organized in a tree. Each bin has an associated size class and stores/manages 
regions of this size class. A bin's regions are managed and accessed through the bin's runs. Each bin 
has a member element representing the most recently used run of the bin, called 'current run' with 
the variable name runcur. A bin also has a tree of runs with available/free regions. This tree is used 
when the current run of the bin is full, that is it doesn't have any free regions. The bin structure is 
portrayed in Figure 5. 
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Figure 5: Bins [arena_bin_t) 


Figure B summarizes our technical overview of the jemalloc architecture. 



Figure B: Architecture of jemalloc 
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Exploitation Primitives 

Before we start our analysis we would like to point out that jemalloc (as well as other malloc 
implementations) does not implement concepts like 'unlinking' or 'frontlinking' which have proven to be 
catalytic for the exploitation of dlmalloc and Microsoft Windows allocators. That said, we would like to 
stress the fact that the attacks we are going to present do not directly achieve a write-4-anywhere 
primitive. We, instead, focus on how to force malloc() (and possibly realloc()) to return a chunk that 
will most likely point to an already initialized memory region, in hope that the region in question may 
hold objects important for the functionality of the target application (C++ VPTRs, function pointers, 
buffer sizes and so on). Considering the various anti-exploitation countermeasures present in modern 
operating systems (ASLR, DEP and so on), we believe that such an outcome is far more useful for an 
attacker than a 4 byte overwrite. 

It is our goal to cover all possible cases of data or metadata corruption, specifically: 

1. Adjacent region overwrites 

2. Run header corruptions 

3. Chunk header corruptions 

4. Magazine (a.k.a. thread cache) corruptions are not covered in this whitepaper since Mozilla 
Firefox does not use thread caching. For more information on this subject please see [PHRC] 
and (PHRK). 

Adjacent Region Overwrites 

The main idea behind adjacent heap item corruptions is that you exploit the fact that the heap 
manager places user allocations next to each other contiguously without other data in between. In 
jemalloc regions of the same size class are placed on the same bin. In the case that they are also 
placed on the same run of the bin then there are no inline metadata between them. Therefore, we 
can place a victim object/structure of our choosing in the same run and next to the vulnerable 
object/structure we plan to overflow. The only requirement is that the victim and vulnerable objects 
need to be of a size that puts them in the same size class and therefore possibly in the same run. 
Since there are no metadata between the two regions, we can overflow from the vulnerable region to 
the victim region we have chosen. Usually the victim region is something that can help us achieve 
arbitrary code execution, for example function pointers. 

In order to be able to arrange the jemalloc heap in a predictable state we need to understand the 
allocator's behavior and use heap manipulation tactics to influence it to our advantage. In the context 
of browsers, heap manipulation tactics are usually referred to as 'Heap Feng Shui' after Alexander 
Sotirov's work (FENG). By 'predictable state' we mean that the heap must be arranged as reliably as 
possible in a way that we can position data where we want. This enables us to use the tactic of 
corrupting adjacent regions, but also to exploit use-after-free bugs. In use-after-free bugs a memory 
region is allocated, used, freed and then used again due to a bug. In such a case if we know the 
region's size we can manipulate the heap to place data of our own choosing in the freed region's 
memory slot on its run before it is used again. Upon its subsequent incorrect use the region now has 
our data that can help us hijack the flow of execution. 

To explore jemalloc's behavior and manipulate it into a predictable state we use an algorithm similar 
to the one presented in (HQEJ). Since in the general case we cannot know beforehand the state of 
the runs of the class size we are interested in, we perform many allocations of this size hoping to 
cover the holes (i.e. free regions) in the existing runs and get a fresh run. Hopefully the next series of 
allocations we will perform will be on this fresh run and therefore will be sequential. As we have seen, 
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sequential allocations on a largely empty run are also contiguous. Next, we perform such a series of 
allocations controlled by us. In the case we are trying to use the adjacent regions corruption tactic, 
these allocations are of the victim object/structure we have chosen to help us gain code execution 
when corrupted. The following step is to deallocate every second region in this last series of 
controlled victim allocations. This will create holes in between the victim objects/structures on the 
run of the size class we are trying to manipulate. Finally, we trigger the heap overflow bug forcing, 
due to the state we have arranged, jemalloc to place the vulnerable objects in holes on the target run 
overflowing into the victim objects. We use and elaborate on this approach in the following 
paragraphs while discussing a case study on the Mozilla Firefox browser. 

Run Header Corruptions 

In a heap overflow situation it is pretty common for the attacker to be able to overflow a memory 
region which is not followed by other regions (like the wilderness chunk in dlmalloc, but in jemalloc 
such regions are not that special). In such a situation, the attacker will most likely be able to 
overwrite the run header of the next run. Since runs hold memory regions of equal size, the next 
page aligned address will either be a normal page of the current run, or will contain the metadata 
(header) of the next run which will hold regions of different size (larger or smaller, it doesn't really 
matter). In the first case, overwriting adjacent regions of the same run is possible and thus an 
attacker can use the techniques that were previously discussed. The latter case is the subject of the 
following paragraphs. 

People already familiar with heap exploitation, may recall that it is pretty common for an attacker to 
control the last heap item (region in our case) allocated, that is the most recently allocated region is 
the one being overflown. This allows an attacker to corrupt a run's header. When a run's metadata 
are overwritten, the 'bin' pointer can be made to point to a fake bin structure. This is not a good idea 
because of two reasons. First, the attacker needs further control of the target process in order to 
successfully construct a fake bin header somewhere in memory. Secondly, and most importantly, the 
'bin' pointer of a region's run header is dereferenced only during deallocation. A careful study of the 
jemalloc source code reveals that only 'run->bin->regO_offset' is actually used (somewhere in 
'arena_run_reg_dalloc()'), thus, from an attacker's point of view, the bin pointer is not that interesting 
('regO_offset' overwrite may cause further problems as well leading to crashes and a forced interrupt 
of our exploit). 

Our attack consists of the following steps. The attacker overflows the last item of a run (for example 
run #1) and overwrites the next run's (e.g. run #2) header. Then, upon the next malloc() of a size 
equal to the size serviced by run #2, the user will get as a result a pointer to a memory region of the 
previous run (run #1 in our example). It is important to understand that in order for the attack to 
work, the overflown run should serve regions that belong to any of the available bins. 

Chunk Header Corruptions 

We will now focus on what the attacker can do once she is able to corrupt the chunk header of an 
arena. Although the probability of directly affecting a nearby arena is low, a memory leak or the 
indirect control of the heap layout by continuous bin-sized allocations can render the technique 
described in this section a useful tool in the attacker's hand. The scenario we will be analyzing is the 
following: The attacker forces the target application to allocate a new arena by controlling its heap 
allocations. She then triggers the overflow in the last region of the previous arena (the region that 
physically borders the new arena) thus corrupting the chunk header metadata. When the application 
calls 'free()' on any region of the newly allocated arena, the jemalloc housekeeping information is 
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altered. On the next call to 'mallocf)', the allocator will return a region that points to already allocated 
space of (preferably] the previous arena. 

Case Study: Mozilla Firefox 

□ur jemalloc debugging tool, unmaskjemalloc, is implemented using the Python scripting support of 
the GNU Debugger (gdb). While unmaskjemalloc supports as-is Linux 32-bit and 64-bit Mozilla 
Firefox targets, there is a problem when it comes to the Mac OS X operating system. Apple's gdb is 
based on the 6.x gdb tree, which means that it does not have support for Python scripting. New gdb 
development snapshots support Mach-0 binaries, but cannot load Apple's fat binaries. In order to 
solve this problem we use Apple's lipo utility and a script we have developed called lipodebugwalk.py. 
This script recursively uses Apple's lipo on the binaries of Firefox.app. Moreover, lipodebugwalk.py 
also has support for Mozilla Firefox's debug symbol binary files. Figure 7 includes the output of using 
fetch-symbols, py (provided by Mozilla) to get debug symbols for Firefox, and the use of 
lipodebugwalk.py to allow a custom-compiled version of gdb to load Firefox. 



Figure 7: Example use of fetch-symbols.py and lipodebugwalk.py 


The above procedure allows us to utilize unmaskjemalloc to explore how jemalloc manages Firefox's 
heap and aid us in the process of exploit development. Figure 8 portrays the help message of 
unmaskjemalloc and shows the functionality we have implemented in the tool: 
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Figure 8: Functionality of the unmaskJemalloc utility 


Using unmaskjemalloc we can investigate how we can manipulate Firefox's jemalloc-managed heap 
from Javascript. The following script uses unescaped strings and arrays to demonstrate controlled 
allocations and deallocations. Since Firefox implements mitigations against traditional heap spraying, 
the script uses random padding to the allocated blocks [CORL]. 

<html> 

<head> 

<script> 

function jemalloc_spray(blocks, size) 

{ 

var blocksize = size / 2; 

// rop/bootstrap/whatever 

var marker = unescapef"%ubeef%udead"); 

marker += marker; 

// shellcode/payload 

var content = unescapef"%u6666%u6666"); 

whilefcontent.length < (blocksize / 2)) 

{ 

content += content; 

} 

var arr = [ ]; 

for(i = 0; i < blocks; i++) 

{ 

// construct the random block padding 
var rndl = Math.floor(Math.random() * 1000) % 16; 

var rnd2 = Math.floor(Math.random() * 1000) % 16; 

var rnd3 = Math.floor(Math.random() * 1000) % 16; 

var rnd4 = Math.floor(Math.random() * 1000) % 16; 

var rndstr = "%u" + rndl.toString() + rnd2.toString(); 
rndstr += "%u" + rnd3.toString() + rnd4.toString(); 

var padding = unescape(rndstr); 

while(padding.length < blocksize - marker.length - content.length) 

{ 
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padding += padding; 

} 

// construct the block 

var block = marker + content + padding; 

// if required repeat the block 
while(block.length < blocksize) 

{ 

block += block; 

} 

// spray block 

arr[i] = block.substr(0); 


Math.asin(l); 

marker = unescape("%ubabe%ucafe"); 
marker += marker; 

content = unescape("%u7777%u7777"); 

while(content.length < (blocksize / 2)) 

{ 

content += content; 

} 

for(i = 0; i < blocks; i += 2) 

{ 

delete(arr[i]); 

// arr.splice(i, 1); 
arr[i] = null; 


var ret = trigger_gc(); 

alert("After garbage collection: " + ret.length); 

for(i = 0; i < blocks; i += 2) 

{ 

var rndl = Math.floor(Math.random() * 1000) % 16; 

var rnd2 = Math.floor(Math.random() * 1000) % 16; 

var rnd3 = Math.floor(Math.random() * 1000) % 16; 

var rnd4 = Math.floor(Math.random() * 1000) % 16; 

var rndstr = "%u" + rndl.toString() + rnd2.toString(); 
rndstr += "%u" + rnd3.toString() + rnd4.toString(); 

var padding = unescape(rndstr); 

while(padding.length < blocksize - marker.length - content.length) 

{ 

padding += padding; 

} 

var block = marker + content + padding; 
while(block.length < blocksize) 
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{ 

block += block; 

} 

arr[i] = block.substr(0); 

} 

Math.atan2(6, 6); 
return arr; 

} 

function trigger gcO 

{ 

// force garbage collection 
var gc = []; 

for(i = 0; i < 100000; i++) 

{ 

gc[i] = new ArrayO ; 

} 

return gc; 

} 

function run sprayO 

{ 

// 1024 spray blocks of size 320 (target run: 512) 
var foo = jemalloc_spray(1024, 320); 

alert(foo.length); 

} 

</script> 

</head> 

<body onload="run_spray();"> 

0x1337 

</body> 

</html> 

Conclusion 

In this whitepaper we have analyzed the jemalloc memory allocator from an exploitation perspective. 
We have developed exploitation primitives that can be used to attack any application that utilizes 
jemalloc. Moreover, we have applied these primitives to the most widely used jemalloc application, 
namely the Mozilla Firefox browser. Our unmaskjemalloc debugging utility can be used during exploit 
development to explore the internals of jemalloc and help the researchers willing to continue our 
work. 
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